Video Highlights Detection and Summarization with Lag-Calibration based on Concept-Emotion Mapping of Crowd-sourced Time-Sync Comments

摘要

With the prevalence of video sharing, there are increasing demands forautomatic video digestion such as highlight detection. Recently, platforms withcrowdsourced time-sync video comments have emerged worldwide, providing a goodopportunity for highlight detection. However, this task is non-trivial: (1)time-sync comments often lag behind their corresponding shot; (2) time-synccomments are semantically sparse and noisy; (3) to determine which shots arehighlights is highly subjective. The present paper aims to tackle thesechallenges by proposing a framework that (1) uses concept-mapped lexical-chainsfor lag calibration; (2) models video highlights based on comment intensity andcombination of emotion and concept concentration of each shot; (3) summarizeeach detected highlight using improved SumBasic with emotion and conceptmapping. Experiments on large real-world datasets show that our highlightdetection method and summarization method both outperform other benchmarks withconsiderable margins.